MAGENTA Technology: A Family of Multi-Agent Intelligent Schedulers

The paper describes a family of Intelligent Schedulers (known as i-Schedulers) for a variety of applications based on Magenta agent technology, and characterized by a number of unique, advanced features such as eventdriven, real-time, incremental, continuous scheduling and schedule improvement, scalability from small to very large enterprise networks, pro-active agent negotiations, multi-criteria schedule analysis and rich decision support for the user. Two schedulers from the family have been developed and applied to commercial problems: a road transportation scheduler and an ocean fleet scheduler. The third member of the family, namely a project management scheduler, is in the design stage. 1. Application areas The family of Magenta schedulers covers very diverse scheduling problems encountered in road transportation, ocean fleet scheduling and project management. Each is described, in turn, below. 1.1. Ocean Scheduler Applications This section gives a description of the implementation of a Magenta Multi-Agent i-Scheduler for Tankers UK Ltd, the UK operations centre for Tankers International LLC based in Cyprus. The company manages a fleet of 48 VLCCs (very large crude carriers) which range in size from 260,000mts to 440,000mts. This represents approximately 10% of the world fleet of these huge vessels. Ocean fleet scheduling involves planning multibillion dollar cargoes, where the cost of even the smallest mistake is very high. A single ocean vessel journey may cost half a million dollars and bring to the operating company a return of two millions dollars, or more. The oil transportation market in which Tankers International operates is characterised by the volatile nature of the trade in terms of both demand and price. Most cargoes are announced through the Spot Market and negotiations on price are conducted through multiple brokers with the price being totally dependant on market conditions. A smaller number of cargoes are given through a Contract of Affreightment (COA) where a customer agrees to provide a tanker company with a certain number of cargoes in an agreed period: example could be 8 cargoes of approximately 275,000mts in next 12 months. Oil transportation is a global business with the main load areas for VLCCs being in the Arabian Gulf, West Africa, North Sea and Mediterranean; the main discharge areas being USA, Northern Europe, China and Japan. Cargoes are announced to the market some 14-30 days in advance of the loading date; this introduces a problem for the tanker operator as a VLCC voyage may be as long as 8 weeks from Arabian Gulf to US Gulf and return via Suez Canal. Vessel Operators will therefore anticipate demand and plan to have vessels in a load area for a likely cargo at the right time. As Tankers International has a large fleet there are many more possibilities which can be covered than for a small operator with only a few ships. The following is a list of goals which the fleet operator attempts to achieve as part of the scheduling activity. Some of these are mutually exclusive so a balance needs to be struck: • Ensure that all COA cargoes which have been announced can be carried • Ensure that COA cargoes expected to be announced in the next few days can be carried • Find the most profitable cargo for each ship • Reduce the number of waiting or idle days between cargoes • Reduce the number of days cruising in ballast; that is, increase the laden/ ballast ratio. Due to the size of these vessels there are very few ports (relatively) which can accommodate them and as each vessel has different physical characteristics it is essential that the physical dimensions of the terminal at the port are checked in advance against those of the vessel both in ballast and when loaded with the cargo. It is easily possible for a vessel to be able to enter a port in ballast condition but as loading increases its draught for it not to be able to exit the port! Complex calculations therefore need to be performed to establish the final draught of the vessel after loading, which will depend on the specific gravity of the grade of crude oil being transported and the temperature of the Ocean at the time of loading. The next constraint to consider is the date of loading and the distance and routing to the load port to check that the vessel is able to sail to the load port to arrive in time for the Laycan (the window in time during which loading must take place). Sailing speeds for all vessels need to be stored and precise distances from the port of discharge to the next load port need to be made available to the system from an external source. There are also licences or legal considerations: certain countries will not allow vessels sailing under their flag to enter certain ports, particularly in war torn areas such as Iraq. There are restrictions on vessels entering US ports related to insurance cover, which not all operators will accept. Many ports these days will not accept vessels with single hulls due to the danger of oil spills in the event of a collision. Vetting of vessels is conducted by the Oil majors for safety and seaworthiness; if a vessel fails inspection or if the vetting has expired some customers will not accept the vessel. The rules in this area are complex and constantly changing. Finally there are customer preferences which are a series of ad-hoc rules, but these are more often ‘soft’ rules rather than absolute the customer may accept this vessel if he receives some incentive – usually in the shape of a discount! We shall describe here the planning of a single cargo some 14 – 30 days prior to loading. However, because vessels are delayed in ports, are affected by bad weather – such as hurricanes in the U.S Gulf – and suffer mechanical breakdowns, the plan invariably has to be modified. The Magenta Ocean i-Scheduler is designed to cope with re-planning in a very effective way – by performing only a localised re-planning. This means that only the agents related to the vessel(s) affected are activated and start negotiations to find all possible options. The System makes calculations of profit for each option and presents all options which are achievable to the user ranked by a set of agreed criteria. It is this ability to calculate all possible options accurately and quickly and then rank them that makes the Magenta Ocean i-Scheduler so powerful. Calculating the most profitable option is not simple. The cargo with the highest TCE (profitability) is not always the best option. Such cargos may require a large number of waiting days before loading commences, when the vessel is not being paid. The load port may be on the other side of the world so there is a long ballast leg before loading or the discharge port may be in an area far away from the next possible load port. The ideal scenario for a vessel is a long voyage, for example, from Arabian Gulf to U.S. Gulf (about 11,000 miles) and from there to take a relatively short ballast leg to West Africa (4-6,000 miles) then take a cargo to China (9,500 miles) then return to the AG (6,000 miles) this would give a high number of earning days – a good laden/ ballast ratio. Contrast this with a cargo from the AG to the U.S. West Coast (11,370 miles). As the Panama Canal is too small for a VLCC the only option is to ballast all the way back to the AG as this is the nearest load area. A fully loaded VLCC cannot pass through the Suez Canal but must offload much of its cargo before entering the canal and then pay for the oil to be transported via pipeline to the northern end of the canal – due to the time and expense of this the VLCCs usually sail around the Cape of Good Hope en-route to the U.S. Today ocean fleet operators have to rely on human experts to perform these complex tasks. But the problem with human experts is that they are under constant time pressure and under stress of performing several concurrent tasks. Besides, there is always a high risk of loosing control of a scheduling process if an expert is temporarily unavailable or leaves the company. Human experts also have a limit of how many vessels they can consider at one time and are therefore not capable of extending their knowledge over rapidly growing fleets. One of the most valuable results that were achieved during the implementation of Magenta Ocean i-Scheduler was capturing and refining knowledge of experts with decades of experience in ocean transport scheduling. The capturing and using this expertise enabled Magenta Scheduler to speed up the scheduling process and increase its accuracy. Whilst the best experts required 1-4 hours to plan a single cargo and produce a good schedule, the Ocean i-Scheduler takes around 10 seconds to calculate all feasible schedule options and present them to the operator in a user-friendly manner. Another important result that was achieved is high transparency for operators, customers, brokers and other participants of the workflow process. A major benefit of the system, as confirmed by the operators, is the clarity and usability of the user interface. Before implementing the Magenta Scheduler, scheduling experts had to refer to several screens and charts in order to understand the status of the fleet and to see important events. With the Ocean iScheduler they now have only one screen to refer to. The simplicity of the interface helps the user navigate through the system and easily find all up-dated information on numerous charts, tables and graphs. An example of one of the quantitative results that was achieved during testing is that where a vessel was delayed in port by 2 days during discharge, this meant it would be late for its next planned cargo; all other vessels in the area also had planned cargoes and were steaming in mid-ocean towards their load ports. Experienced operators in two countries grappled with this problem for 4 hours before finally arriving at a possible solution. When all the data was input into the Ocean i-Scheduler it took only 36 seconds to arrive at the same answer. The Magenta Scheduler tracks the entire flow from event to final decision and writes all related data to a new decision database. This database contains the status message, the event, the state of the fleet prior to the event, the options considered to respond to the event, the recommended option according to the company objectives, the decision from the operator (if there is one) and then the resulting state of the schedule. The solution can then also track the actual schedule performance to understand if the scheduling activity was achieved without exception or even rescheduled due to another event. Agents are then used to analyse this data to discover patterns that are causing inefficiency and/or inconsistencies with the best practice. As the data and analysis can be represented by a semantic network and the underlying ontology is in a similar format, this new knowledge can be inserted into the knowledge-base quickly. While the current view from users requires a process with human interaction, it is readily imaginable that more and more automation can accelerate this process of learning and adaptation. 1.2. Road Scheduler Applications The first version of the Logistics i-Scheduler was tested on two sets of real-life data obtained from Client A (3 Party Logistics Provider) and Client B (a provider of logistics and freight management services). Client A Requirements were to create transportation schedules in real time for 200 transportation instructions and 51 trucks (36 own fleet trucks plus 15 third-party carriers) operating on the UK Business Network. The network included 9 Distribution Centres/factories, cross-dock points for primary/secondary moves consolidation, 3 truck bases doing shared operations. In addition to requirements discussed earlier client specified handling transportation instruction availability windows, backhaul, consolidation, vehicle capacity availability windows and constraint stressing. Client B requirements were to create transportation schedules for 4000 transportation instructions and 200 trucks operating on the UK business network. The network included primary and secondary deliveries between about 600 locations, 3 cross docks, 4 secure trailer swap locations and other types of locations. The network was also characterized by considerable number of very small orders. Special requirements included dynamic routing, cross-docking, handling location availability windows and driver breaks. The Logistics i-Scheduler has completed a schedule for 200 transportation instructions in 8 minutes, planning 116 journeys of a total of 20790 miles. The quality compares well with the results of the current process which requires several hours to produce a schedule, and has two operators working on the basis of a plan day 1 for day 3 execution. With the Logistics i-Scheduler it would be feasible to plan day 1 for day 2 execution or even day 1. For 4000 orders with dynamical routing through 3 cross docks it took the Logistics i-Scheduler about 4 hours to build a schedule. . This schedule shows strong consolidation of small orders onto trucks. It is also capable of incrementally planning new orders in near real time (a few seconds for a new order). As far as we know, this has not been achieved by any other transportation scheduling system. Road transportation logistics is amongst the most complex business problems. The complexity is caused by the exceedingly high variety of possible solutions (large solution space), which rules out traditional combinatorial search algorithms, and uncertainty due to high dynamics and volatility of the operational environment and openness of business networks, which makes optimisation impractical – a single optimisation run is typically an order of magnitude longer than a typical interval between two consecutive changes in operational conditions. Resource allocation is an ongoing continuous decision making process in real time where criteria are changing “on the fly”. Therefore the two key capabilities currently required from schedulers are supporting complex business networks and planning in continuous mode. An effective road transportation scheduler must handle transportation instructions (TI) from many different loading points to many different destinations (e.g. customer locations and cross docks where cargoes are offloaded and consolidated) and many different routes by which orders can be delivered. Choosing the best route from the point of view of consolidation or other criteria is referred to as dynamic routing. The scheduler must also be able to allocate cargos of many different sizes and weights to many different types of trucks and trailers; take into account preferences of owners, operators and drivers and fit the schedule into numerous constraints imposed by warehouse working hours, driver work rules, safety regulations and enterprise policies, eg, on choosing between own fleet and third-party carriers. Different companies have different critical constraints, e.g. permission to override time or other constraints to achieve a more efficient schedule. The schedule created must be not only feasible but also efficient, i.e. possibilities for backhauls and consolidations should be found. Complexity is also defined by the number and variety of orders (and other events that affect scheduling) per day and the number and variety of transportation resources such as trucks. In addition, the scheduler is expected to rapidly reschedule orders and transportation resources affected by unexpected events such as: the arrival of new orders, cancellations, failures, bad weather conditions, road works and no-show of drivers or loading crews. To enable enterprises to plan and re-plan continuously, reacting to events in real-time, schedulers must support Planning / Commit / eXecute (PCX) stages and plan across a multi-day planning horizon. An individual truck may be in a Planning stage where it is being assigned orders and a journey is being built for it. During this phase orders can be added or removed as a result of new events and the route taken by the truck can be changed. At some point the scheduler must commit the truck. This will trigger communications to warehouses, driver shift planners, truck servicing etc to make ready the truck for its journey. During this phase changes to the truck schedule are feasible but of course there would be knock on effects in the warehouse, driver planning etc. The eXecute stage is when the driver begins his pre-journey checks and continues until his debriefing at the end of his shift is completed. During this phase a high level of sophistication would be needed to alter the truck schedule in transit. Magenta Logistics i-Scheduler supports PCX by allowing Commit of individual trucks from a rolling schedule. To achieve competitive advantage schedulers must take into account real-time economy where decisions are based not on some average rule but on the detailed analysis of the current situation. For example, a truck loaded by only 10% with a special cargo may be very profitable whilst a rule-based scheduler would not allow a nearly empty truck to start a journey. The requirement is to assess the economy of each truck, each journey, etc., which implies using the activitybased cost model. The current generation of batch schedulers cannot satisfy these requirements; a fundamentally new approach to the task of allocating resources in real-time is therefore needed. Designing a scheduler that can cope with such a variety of operating conditions, handle uncertainty related to the occurrence of events and at the same time continuously produce schedules that maximise the specified value (or minimise transportation costs) is a real intellectual challenge. 1.3. Project Management Applications Projects vary from those that can be implemented by 4-5 persons working in an office to multi-million construction works involving hundreds of people, machinery and materials. Scalability is therefore the key feature. Also, in contrast to a supply chain, projects have clear starting points and completion deadlines and are subject to even larger number of unpredictable events threatening the orderly execution. It is therefore of paramount importance to have a rapid and effective re-scheduling activity to keep project plans up to date. Magenta Project Scheduler is in the design stage at present. 2. Magenta approach to scheduling The software that exists on the market is ignoring true complexity of scheduling in order to automate high volume business processes. Such tools rely on rigid business rules to optimize the process. They are not scalable or robust and often prevent business from growing. Conventional software handles the scheduling process in rigid batch mode, which means that each time batch operation is carried out the already built schedule needs to be completely broken and rebuilt from the very beginning. The Magenta approach to scheduling described in this paper handles complexity by balancing and resolving conflicts of interests of many active players rather than by following given rules and satisfying specified constraints. This is achieved primarily by assigning an autonomous agent to every player in the business process and tasking agents to obtain the best possible deals for their clients. Players here include the enterprise as a whole (represented by the Enterprise Agent) and all individual demands and resources (including crews and resource owners). Demands and resources that have common interests self-organise into groups represented by a single agent. Agents may decide to compete or co-operate depending on prevailing circumstances. Human operators are provided with facilities for monitoring the scheduling process and overruling the decisions made by agents. iSchedulers support users in their interaction with the system by providing them with options and evaluating consequences of their decisions. The scheduling process is event-driven, which means that whenever an event that affects the schedule occurs, agents re-negotiate the allocation of affected resources. This re-scheduling is local, only affected parts of the schedule change, and therefore reacting to events is rapid and disturbance of the schedule is minimised. The scheduling process is also knowledge-driven (rather than rule-based or constraint-driven), which means that agents before acting consult domain knowledge available to them in Ontology. Knowledge on scheduling is separated from the resource allocation mechanism, which greatly simplifies updates and increases the reuse of code. The process of formalising domain knowledge helps in refining it and closing the gaps left due to the empirical nature of knowledge collection. 3. Architecture of i-Schedulers The family of i-Scheduler is based on the standard Magenta software architecture as depicted in Figure 1 and described in [1] and [2]. The blue components (Engine, Virtual Market and Ontology Management Toolkit) are generic and common to all Magenta multi-agent applications, representing approximately 50-60% of the total amount of code. The green components (Virtual Market Extensions, Agent Toolkit, Ontology, Scenes, APIs and User Interfaces) are designed for each application thus customising the scheduler for each client and each scheduling problem. The software is powered by a Multi-Agent Engine which contains all runtime tools necessary for the agent operation. Formalised knowledge is in Ontology supported by a powerful ontology editor. Agents negotiate scheduling decisions in the Virtual Market. The i-Scheduler family is implemented on a J2EE platform. Fig. 1. Architecture of Magenta multi-agent applications.